Real-Time Spoken-Language Programming for Cooperative Interaction with a Humanoid Apprentice

An apprentice is an able-bodied individual that will interactively assist an expert, and through this interaction, acquire knowledge and skill in the given task domain. A humanoid apprentice should have a useful repertoire of sensory-motor acts that the human can command with spoken language, along with a real-time behavioral sequence acquisition ability. The learned sequences should function as executable procedures that can operate in a flexible manner that are not rigidly sensitive to initial conditions. Our study integrates these capabilities in a real-time system on the HRP-2 humanoid, for learning a cooperative assembly task. We previously defined a system for Spoken Language Programming (SLP) that allowed the user to guide the robot through an arbitrary, task relevant, motor sequence via spoken commands, and to store this sequence as re-usable macro. Here, we significantly extend the SPL system: It integrates vision and motion planning into the SLP framework, providing a new level of flexibility in the actions that can be created, and it allows the user to create “generic” functions with arguments (e.g. Give me X), and it allows multiple functions to be created.